We consider the task of learning a classifier for semantic segmentation usingweak supervision in the form of image labels which specify the object classespresent in the image. Our method uses deep convolutional neural networks (CNNs)and adopts an Expectation-Maximization (EM) based approach. We focus on thefollowing three aspects of EM: (i) initialization; (ii) latent posteriorestimation (E-step) and (iii) the parameter update (M-step). We show thatsaliency and attention maps, our bottom-up and top-down cues respectively, ofsimple images provide very good cues to learn an initialization for theEM-based algorithm. Intuitively, we show that before trying to learn to segmentcomplex images, it is much easier and highly effective to first learn tosegment a set of simple images and then move towards the complex ones. Next, inorder to update the parameters, we propose minimizing the combination of thestandard softmax loss and the KL divergence between the true latent posteriorand the likelihood given by the CNN. We argue that this combination is morerobust to wrong predictions made by the expectation step of the EM method. Wesupport this argument with empirical and visual results. Extensive experimentsand discussions show that: (i) our method is very simple and intuitive; (ii)requires only image-level labels; and (iii) consistently outperforms otherweakly-supervised state-of-the-art methods with a very high margin on thePASCAL VOC 2012 dataset.
展开▼
